\documentclass[11pt]{book}

%\RequirePackage[12tabu, orthodox]{nag}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                                                 %
% The mathematical style of these documents is based on the following                                       %
%                                                                                                 %
% A. Thompson and B.N. Taylor. The NIST Guide for the Use of the International System of Units.   %
%    NIST Special Publication 881, 2008.                                                          %
%                                                                                                 %
% http://www.nist.gov/pml/pubs/sp811/index.cfm                                                    %
%                                                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\input{../Bibliography/commoncommands}
\IfFileExists{../Bibliography/gitrevision.tex}
{\input{../Bibliography/gitrevision}}
{\newcommand{\gitrevision}{unknown} }

\begin{document}

\bibliographystyle{unsrt}
\pagestyle{empty}

\begin{minipage}[t][9in][s]{6.5in}

\headerA{1018-4\\Sixth Edition\\}

\headerB{
Fire Dynamics Simulator\\
Technical Reference Guide\\
Volume 4: Software Quality Assurance\\
}

\headerC{
\authortitlesigs
}

\vfill

\headerD{1018}

\vfill

\logosigs

\end{minipage}

\newpage

\hspace{5in}

\newpage

\begin{minipage}[t][9in][s]{6.5in}

\headerA{1018-4\\Sixth Edition\\}

\headerB{
Fire Dynamics Simulator\\
Technical Reference Guide\\
Volume 4: Software Quality Assurance\\
}

\headerC{
\authorsigs
}

\headerD{1018}

\headerC{
\begin{flushright}
\today \\
Revision:~\gitrevision
\end{flushright}}

\vfill

\begin{flushright}
\includegraphics[width=1.2in]{../Bibliography/doc}
\end{flushright}

\titlesigs

\end{minipage}

\newpage

\disclaimer{1018-4}

\frontmatter

\pagestyle{plain}

\input{../Bibliography/authors}


\chapter{Preface}

This is Volume 4 of the FDS Technical Reference Guide. The purpose of this document is to describe the policies and procedures for developing and maintaining the Fire Dynamics
Simulator (FDS) and the companion visualization program, Smokeview. Such a document is commonly referred to as a {\em Software Quality Assurance Plan}, known through the rest of this document as ``the Plan.'' This document will be updated as the necessity arises and will establish and provide the basis for a uniform and concise standard of practice for FDS and Smokeview development.

This plan is based in part on IEEE Standard 828~\cite{IEEE-828} and ASTM~E1355~\cite{ASTM:E1355}. The volumes that make up the FDS Technical Reference Guide are based in part on ASTM E 1355, {\em Standard Guide for Evaluating the Predictive Capability of
Deterministic Fire Models}~\cite{ASTM:E1355}. ASTM~E~1355 outlines the process of assessing the accuracy of a fire model. Volumes~1 through 3 are the result of this evaluation process. A model such as FDS cannot remain static. As progress is made in fire science, the model needs to be updated and improved, but still shown to reliably predict the kinds of phenomena for which it was originally designed. The process to ensure this reliability includes

\begin{itemize}
    \item Relevant documentation for the model: Volume 1 describes the mathematical model and numerical method. Volumes~2 and 3 document past and present model verification and validation work, respectively. Instructions for using FDS are contained in a separate User's Guide~\cite{FDS_Users_Guide}.

    \item Description of the development process for the model.

    \item Control and tracking of the source code using a centralized project repository on GitHub.

    \item An automated build, verification, validation, and regression testing process then helps the FDS development team by performing automatic error checking and collecting performance and accuracy statistics between FDS revisions.

    \item Details of the review process used by the developers and NIST to ensure the quality of the model and related publications.

\end{itemize}

This document applies only to the program FDS, its companion visualization program called Smokeview,  the related NIST and VTT publications and Internet
sites, and the various utility programs that support FDS and Smokeview. This document does not apply to third-party software developed by others
not affiliated with NIST, VTT, or its software development collaborators.


% \chapter{Disclaimer}
\input{../Bibliography/disclaimer}


\chapter{Acknowledgments}

\label{acksection}

The idea for this Guide originated with Jason Floyd, a co-developer of FDS and practicing engineer at Jensen Hughes. He readily saw the need for a clear description of the process of developing and maintaining FDS and Smokeview. Bryan Klein, currently with Thunderhead Engineering, Inc., instituted a number of the internet tools that are described in the Guide, including the source code repository, issue tracking system, and discussion group sites.


\newpage

\tableofcontents

\mainmatter


\chapter{Overview of FDS and Smokeview}

This document describes the processes used during the development and deployment of the Fire Dynamics Simulator (FDS) and its companion program Smokeview.  This software quality assurance (SQA) plan is intended to guide the planning for modifications to the model, provide required reviews for both software and associated documentation of the model, define testing to be conducted prior to the release of an updated model, describe problem reporting and resolution procedures, and ensure all records, source code, and released software is kept available for the life of the code.  While this report and many of our practices follow the ASTM International Standard 1355-05, ``Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models'' \cite{ASTM:E1355} which has been used extensively to guide the documentation, verification, and validation of the model, other standards have been followed as well.  Most notably, Institute of Electrical and Electronics Engineers (IEEE) standards IEEE 730-2002 \cite{IEEE:730} and 828~\cite{IEEE-828} define best practices in important areas of software quality assurance and guides the structure and content of this plan.

The volumes that make up the FDS documentation are based in part on ASTM E 1355, {\em Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models}~\cite{ASTM:E1355}. ASTM~E~1355 outlines the process of assessing the accuracy of a fire model. Volumes~1 through 3 are the result of this evaluation process. The main purpose of the present volume is to describe the process by which the model software is developed and maintained. A model such as FDS cannot remain static. As progress is made in fire science, the model needs to be updated and improved, but still shown to reliably predict the kinds of phenomena for which it was originally designed.

The SQA process and requirements outlined in this document apply specifically to the FDS and Smokeview and is focused on ensuring the quality of the numerical predictions of the model.  Any user interface that may be used to develop input for the model is not included in this process. Users must ensure that the input files developed for the simulations accurately reflect the desired model inputs, whether developed using a third-party interface or manually input with a spreadsheet or text editor program.  Documentation of these inputs is included as part of the model documentation outlined below.

This chapter provides a brief overview of the Fire Dynamics Simulator (FDS) and Smokeview, including its history, developers, features, and applications.


\section{Model Type}

FDS is a Computational Fluid Dynamics (CFD) model of fire-driven fluid flow. The model numerically solves a form of the Navier-Stokes equations
appropriate for low-speed, thermally-driven flow with an emphasis on smoke and heat transport from fires. The partial derivatives in the conservation
equations for mass, momentum, and energy are approximated by finite differences, and the solution is updated in time on a three-dimensional,
rectilinear grid. Thermal radiation is computed using a finite volume technique on the same grid as the flow solver. Lagrangian particles are used to
simulate sprinkler droplets, fuel sprays, and unresolvable subgrid objects.

Smokeview is a companion program to FDS that produces images and animations of the results. In recent years, its developer, Glenn Forney, has added
to Smokeview the ability to visualize fire and smoke in a fairly realistic way. In a sense, Smokeview now is, via its three-dimensional renderings,
an integral part of the physical model, as it allows one to assess the visibility within a fire compartment in ways that ordinary scientific
visualization software cannot.

Although not part of the FDS/Smokeview suite maintained at NIST (and not subject to this Plan), there are several third-party and proprietary ``add-ons'' to FDS either available
commercially or privately maintained by individual users. Most notably, there are several Graphical User Interfaces (GUIs) that can be used to create
the input file containing all the necessary information needed to perform a simulation.



\section{Model Version}

Version 1 of FDS was publicly released in February 2000, version 2 in December 2001, version 3 in November 2002, version 4 in July 2004, version 5 in October 2007, and version 6 in October 2013.

Starting with FDS 5, a formal revision management system was implemented to track changes to the FDS source code. The open-source program development tools are provided by GitHub (https://github.com/).

Wiki files, located on the FDS/Smokeview web site, maintained by the developers, details the changes made for each release: \\ \href{https://github.com/firemodels/fds/wiki/Release-Notes}{\textct{https://github.com/firemodels/fds/wiki/Release-Notes}} \\ \href{https://github.com/firemodels/smv/wiki/Release-Notes}{\textct{https://github.com/firemodels/smv/wiki/Release-Notes}}.


\section{Model Developers}


Currently, FDS is developed by the NIST Engineering Laboratory, Fire Research Division, in cooperation with VTT Technical Research Centre of Finland. The developers at NIST and VTT have formed a loose collaboration of interested stakeholders, including:
\begin{itemize}
\item The various contributors listed at the start of this report,
\item The Society of Fire Protection Engineers (SFPE) which conducts training classes on the use of FDS,
\item Fire protection engineering firms that use the software,
\item Some recipients of external grants awarded by the NIST Fire Research Division,
\item Engineering departments at various universities with a particular emphasis on fire.
\end{itemize}


\section{Intended Uses}

Throughout its development, FDS has been aimed at solving practical fire problems in fire protection engineering, while at the same time providing a
tool to study fundamental fire dynamics and combustion. FDS can be used to model the following phenomena:
\begin{itemize}
\setlength{\itemsep}{0.0in}
\item Low speed transport of heat and combustion products from a fire
\item Radiative and convective heat transfer between the gas and solid surfaces
\item Pyrolysis
\item Flame spread and fire growth
\item Sprinkler, heat detector, and smoke detector activation
\item Sprinkler sprays and suppression by water or other agents
\end{itemize}
Although FDS was designed specifically for fire simulations, it can be used for other low-speed fluid flow simulations that do not necessarily include fire or thermal effects. To date, about half of the applications of the model have been for design of smoke control systems and sprinkler/detector activation studies. The other half consist of residential and industrial fire reconstructions.


\section{Input Parameters}

All of the input parameters required by FDS to describe a particular scenario are conveyed via a single text file created by the user. The file contains information about the numerical grid, ambient environment, scenario geometry, material properties, combustion kinetics, and desired output quantities. The numerical grid consists of one or more rectilinear meshes with (usually) uniform cells. All geometric features of the scenario must conform to this numerical grid. Objects smaller than a single grid cell are either approximated as a single blocked cell or cell face, or are rejected. The scenario geometry is input as a series of rectangular blocks. Boundary conditions are applied to solid surfaces as rectangular patches. Materials are defined by their thermal conductivity, specific heat, density, thickness, and burning behavior. There are various ways that this information is conveyed, depending on the desired level of detail.

Any simulation of a real fire scenario involves specifying material properties for the walls, floor, ceiling, and furnishings. FDS treats all of these objects as multi-layered solids; thus, the physical parameters for many real objects can only be viewed as approximations to the actual properties. Describing these materials in the input file is the single most challenging task for the user. Thermal properties such as conductivity, specific heat, density, and thickness can be found in various handbooks, or in manufacturers literature, or from bench-scale measurements. The burning behavior of materials at different heat fluxes is more difficult to describe, and the properties more difficult to obtain. Even though entire books are devoted to the subject~\cite{Babrauskas:2}, it is still difficult to find information on a particular item.

A significant part of the FDS input file directs the code to output various quantities in various ways. Much like in an actual experiment, the user must decide before the calculation begins what information to save. There is no way to recover information after the calculation is over if it was not requested at the start.

A complete description of the input parameters required by FDS can be found in the FDS User's Guide~\cite{FDS_Users_Guide}.


\section{Output Quantities}

FDS computes the temperature, density, pressure, velocity, chemical composition, and various other quantities within each numerical grid cell at each discrete time step. There are typically hundreds of thousands to millions of grid cells and thousands to hundreds of thousands of time steps. In addition, FDS computes at solid surfaces the temperature, heat flux, mass loss rate, and various other quantities. Even though only a small fraction of the computed information can be saved, the output typically consists of fairly large data files. Typical output quantities for the gas phase include:
\begin{itemize}
\setlength{\itemsep}{0.0in}
\item Gas temperature
\item Gas velocity
\item Gas species concentration (water vapor, CO$_2$, CO, N$_2$)
\item Smoke concentration and visibility estimates
\item Pressure
\item Heat release rate per unit volume
\item Gas density
\item Water droplet mass per unit volume
\end{itemize}
On solid surfaces, FDS predicts additional quantities associated with the energy balance between gas and solid phase, including:
\begin{itemize}
\setlength{\itemsep}{0.0in}
\item Surface and interior temperature
\item Heat flux, both radiative and convective
\item Burning rate
\item Water droplet mass per unit area
\end{itemize}
Global quantities recorded by the program include:
\begin{itemize}
\setlength{\itemsep}{0.0in}
\item Total Heat Release Rate (HRR)
\item Sprinkler and detector activation times
\item Mass and energy fluxes through openings or solids
\end{itemize}
Time histories of various quantities at a single point in space or global quantities like the fire's heat release rate (HRR) are saved in simple, comma-delimited text files that can be plotted using a spreadsheet program. However, most field or surface data are visualized with a program called Smokeview, a tool specifically designed to analyze data generated by FDS. FDS and Smokeview are used in concert to model and visualize fire phenomena. Smokeview performs this visualization by presenting animated tracer particle flow, animated contour slices of computed gas variables and animated surface data. Smokeview also presents contours and vector plots of static data anywhere within a scene at a fixed time. A complete list of FDS output quantities and formats is given in Ref.~\cite{FDS_Users_Guide}. Details on the use of Smokeview are found in Ref.~\cite{Smokeview_Users_Guide}.




\section{Governing Equations, Assumptions and Numerics}

The following is a brief description of the major components of FDS. Detailed information regarding the assumptions and governing equations associated with the model is provided in Volume~1 of the FDS Technical Reference Guide~\cite{FDS_Tech_Guide}.
\begin{description}
\item[Hydrodynamic Model] FDS solves numerically a form of the Navier-Stokes equations appropriate for low-speed, thermally-driven flow with an emphasis on smoke and heat transport from fires. The core algorithm is an explicit predictor-corrector scheme that is second order accurate in space and time. Turbulence is treated by means of Large Eddy Simulation (LES). It is possible to perform a Direct Numerical Simulation (DNS) if the underlying numerical grid is fine enough. LES is the default mode of operation.
\item[Combustion Model] For most applications, FDS uses a combustion model based on the mixing limited, infinitely fast reaction of lumped species. Lumped species are reacting scalar quantities that represent a mixture of species such as air which is a mixture of nitrogen, oxygen, water vapor, and carbon dioxide. As with FDS 5, the reaction of fuel and oxygen is not necessarily instantaneous and complete, and there are several optional schemes that are designed to predict the extent of combustion in under-ventilated spaces. The mass fractions of all primitive species can be obtained from a linear combination of the lumped species.
\item[Radiation Transport] Radiative heat transfer is included in the model via the solution of the radiation transport equation for a gray gas. In a limited number of cases, a wide band model can be used in place of the gray gas model to provide a better spectral accuracy. The radiation equation is solved using a technique similar to a finite volume method for convective transport, thus the name given to it is the Finite Volume Method (FVM). Using approximately 100 discrete angles, the finite volume solver requires about 20~\% of the total CPU time of a calculation, a modest cost given the complexity of radiation heat transfer.  Water droplets can absorb and scatter thermal radiation. This is important in cases involving mist sprinklers, but also plays a role in all sprinkler cases. The absorption and scattering coefficients are based on Mie theory. The scattering from the gaseous species and soot is not included in the model.
\item[Geometry] FDS approximates the governing equations on one or more rectilinear grids. The user prescribes rectangular obstructions that are forced to conform with the underlying grid.
\item[Boundary Conditions] All solid surfaces are assigned thermal boundary conditions, plus information about the burning behavior of the material. Heat and mass transfer to and from solid surfaces is usually handled with empirical correlations, although it is possible to compute directly the heat and mass transfer when performing a Direct Numerical Simulation (DNS).
\item[Sprinklers and Detectors] The activation of sprinklers and heat and smoke detectors is modeled using fairly simple correlations of thermal inertia for sprinklers and heat detectors, and transport lag for smoke detectors. Sprinkler sprays are modeled by Lagrangian particles that represent a sampling of the water droplets ejected from the sprinkler.
\end{description}


\section{Limitations}

Although FDS can address most fire scenarios, there are limitations in all of its various algorithms. Some of the more prominent limitations of the
model are listed here.
\begin{description}
\item[Low Speed Flow Assumption] The use of FDS is limited to low-speed\footnote{Mach numbers less than about 0.3} flow with an emphasis on smoke and heat transport from fires. This assumption rules out using the model for any scenario involving flow speeds approaching the speed of sound, such as explosions, choke flow at nozzles, and detonations.
\item[Rectilinear Geometry] The efficiency of FDS is due to the simplicity of its rectilinear numerical grid and the use of a fast, direct solver for the pressure field. This can be a limitation in some situations where certain geometric features do not conform to the rectangular grid, although most building components do. For most practical large-scale simulations, the increased grid resolution afforded by the fast pressure solver offsets the approximation of a curved boundary by small rectangular grid cells.
\item[Fire Growth and Spread] Because the model was originally designed to analyze industrial-scale fires, it can be used reliably when the heat release rate (HRR) of the fire is specified and the transport of heat and exhaust products is the principal aim of the simulation. In these cases, the model predicts flow velocities and temperatures to an accuracy within 10~\% to 20~\% of experimental measurements, depending on the resolution of the numerical grid\footnote{It is rare to find measurements of local velocities and/or temperatures from fire experiments that have reported error estimates that are less than 10~\%. Thus, the most accurate calculations using FDS do not introduce significantly greater errors in these quantities than the vast majority of fire experiments.}. However, for fire scenarios in which the heat release rate is {\em predicted} rather than {\em specified}, the uncertainty of the model is larger. There are several reasons for this: (1) properties of real materials and real fuels are often unknown or difficult to obtain, (2) the physical processes of combustion, radiation and solid phase heat transfer are more complicated than their mathematical representations in FDS, (3) the results of calculations are sensitive to both the numerical and physical parameters. Current research is aimed at improving this situation, but it is safe to say that modeling fire growth and spread will always require a higher level of user skill and judgment than that required for modeling the transport of smoke and heat from specified fires.
\item[Combustion] For most applications, FDS uses a mixing-controlled, lumped species-based combustion model. Lumped species are reacting scalar quantities that represent mixtures of gas species.  For most applications, the lumped species are air, fuel, and combustion products.  In its simplest form, the model assumes that combustion is mixing-controlled, and that the reaction of fuel and oxygen is infinitely fast, regardless of the temperature. For large-scale, well-ventilated fires, this is a good assumption. However, if a fire is in an under-ventilated compartment, or if a suppression agent like water mist or CO$_2$ is introduced, fuel and oxygen are allowed to mix and not burn, according to a few empirically-based criteria. The physical mechanisms underlying these phenomena are complex, and are tied closely to the flame temperature and local strain rate, neither of which are calculated in a large scale fire simulation. Subgrid-scale modeling of gas phase suppression and extinction is still an area of active research. Until reliable models can be developed for building-scale fire simulations, simple empirical rules can be used that prevent burning from taking place when the atmosphere immediately surrounding the fire cannot sustain combustion.
\item[Radiation] Radiative heat transfer is included in the model via the solution of the radiation transport equation (RTE) for a gray gas, and in some limited cases using a wide band model.  The RTE is solved using a technique similar to finite volume methods for convective transport, thus the name given to it is the Finite Volume Method (FVM). There are several limitations of the model. First, the absorption coefficient for the smoke-laden gas is a complex function of its composition and temperature. Because of the simplified combustion model, the chemical composition of the smokey gases, especially the soot content, can effect both the absorption and emission of thermal radiation.  Second, the radiation transport is discretized via approximately 100 solid angles, although the user may choose to use more angles. For targets far away from a localized source of radiation, like a growing fire, the discretization can lead to a non-uniform distribution of the radiant energy. This error is called ``ray effect'' and can be seen in the visualization of surface temperatures, where ``hot spots'' show the effect of the finite number of solid angles. The problem can be lessened by the inclusion of more solid angles, but at a price of longer computing times. In most cases, the radiative flux to far-field targets is not as important as those in the near-field, where coverage by the default number of angles is much better.
\end{description}











\chapter{Configuration Management}

\section{Project Management}

FDS is developed by the NIST Engineering Laboratory, Fire Research Division, in cooperation with VTT Technical Research Centre of Finland. For all projects at the NIST Engineering Laboratory, a designated project leader is responsible for directing and prioritizing model development, error correction, and preparation of documentation for the model development.

Members of the FDS and Smokeview team share in the development and maintenance responsibilities, which include:
\begin{enumerate}
\item Developing new algorithms and improving program functionality
\item Answering user questions
\item Responding to bug reports
\item Issuing periodic updates to the officially released versions of the programs
\item Maintaining the Technical Reference and Users Guides
\item Maintaining a suite of sample cases to demonstrate model use
\item Maintaining a suite of verification and validation cases to test model accuracy and reliability
\item Developing and updating a ``Roadmap'' of future development
\end{enumerate}
Most decisions concerning these various tasks are made individually or by consensus of the team members. In the event of a disagreement over a technical issue, the resolution is facilitated by the principal developers at NIST and VTT. Non-technical issues are addressed by the chief of the Fire Research Division at NIST or the respective counterpart at VTT. All policies and procedures described in this document are subject to review by NIST with input from VTT, SFPE, and other Federal agencies.

Review and approval of software and documentation is part of the standard review process for any report or other product developed by NIST. A minimum of five reviews are required prior to release of reports or software, including two independent technical peer reviews, technical and management reviews at the technical workgroup and division level, and a policy review at the NIST-wide level.  This review is documented and recorded on the NIST standard form NIST 114 along with official approval notification provided to the primary author of the product.

FDS/Smokeview is distributed exclusively through a NIST website dedicated to the model (https://pages.nist.gov/fds-smv/).  Content of the website is the responsibility of the FDS project leader. Additions and changes to the website are made through the same SQA process documented in this plan.


\section{Document Identification and Control}

Document identification and control consists of placing all project files in a central location and maintaining a record of changes to those files.
The central location is known as the {\em Repository}.

Starting in 2007 with FDS version 5, the FDS development team adopted GoogleCode, a free Internet service designed to assist open source software development by providing a repository for source code, revision control, program distribution, bug tracking, and various other very useful services. In 2015, Google terminated this service, and provided all of its projects a migration path to GitHub, a similar free service that uses Git, rather than Subversion, as its version control software.

The FDS manuals are typeset using \LaTeX, specifically, PDF \LaTeX. The \LaTeX\ files are essentially text files that are under version control. The figures are either in the form of PDF or png files, depending on whether they are vector or raster format. There are a variety of \LaTeX\ packages available, including MiKTeX. The FDS developers edit the manuals as part of the day to day upkeep of the model. Different editions of the manuals are distinguished by a specific Git hash tag.


\subsection{Project Repository}

Currently, FDS and Smokeview make use of GitHub, a free service to support software development for open source applications. GitHub uses Git version control software. Under this system a centralized repository containing all project files resides on an external server.   A record of version number when a specific file was last changed is maintained. As an open source program, any individual can obtain a copy of the repository or retrieve specific versions of the repository.

Both FDS and Smokeview live within a GitHub \emph{organization} called ``Fire Models''.  The current location of the organization is \href{https://github.com/firemodels}{\textct{https://github.com/firemodels}}.  The repositories that are used by the FDS and Smokeview projects contain more than 6000 configuration items and are listed below along with a brief description:

\vskip\baselineskip
\begin{tabular}{ll}
fds & FDS source code, verification and validation tests, wikis, and documentation \\
smv & Smokeview source code, integration tests, and documentation \\
exp & Repository of experimental data for model validation \\
out & FDS output results for validation \\
bot & Firebot (continuous integration system) source \\
fds-smv & Web page html source
\end{tabular}
\vskip\baselineskip

Each member of the FDS/Smokeview development team has an account and password access~\footnote{Access to the FDS/Smokeview repositories is by means of two-factor identification.} to the FDS/Smokeview repository. In addition, anonymous users can ``fork,'' or copy, the repository and receive the latest versions of the source code, manuals, and other items. Only Team Members described in the previous chapter can commit changes to the repository. All others may make changes to their own copy of the repository and can suggest these changes to the FDS/Smokeview developers by means of a ``pull request.'' which is then reviewed by the development team for inclusion into the repository.

In the event of an unexpected change to the GitHub service and/or the FDS repository, each member of the development team, plus interested third parties, has a copy of the repository on their local computer. At NIST, the staff computers are regularly backed up as well. Thus, there is very little chance that the project repository will be lost. If need be, the files could be moved to another open source software development site.

Additional details on the use of GitHub for the models are available on the wiki pages for the models at \href{https://github.com/firemodels/fds-smv/wiki}{\textct{https://github.com/firemodels/fds-smv/wiki}}.

\subsection{Version Identification and Numbering}

At the start of a simulation, FDS writes header information to the Smokeview output file, FDS output file, and the FDS log file.  This header information contains the version of FDS used to perform that simulation.

Official versions of the model released by NIST are identified with specific version numbers tied to specific revisions in the repository and are identified on the download page for the model that can be compared to the version output from the model. For example, the following information is included on the FDS download website for FDS version 6.5.2

\begin{lstlisting}
 Version          : FDS 6.5.2
 Revision         : Git-r21-0-g4e9103f
 Revision Date    : Wed Aug 24 16:29:46 2016 -0400
 Compilation Date : Aug 24, 2016  22:28:55
\end{lstlisting}

New versions of FDS and Smokeview are identified using a specific numbering convention consisting of three integers separated by periods, where the first number is the {\em major} release, the second is the {\em minor} release, and the third is the {\em maintenance} release. For example, FDS 6.5.3 indicates that this is FDS 6, the sixth major release. The 5 indicates a significant upgrade (a minor release), but still within the framework of FDS 6.  The 3 indicates the third maintenance release of 6.5, mostly bug fixes and minor user requests. All volumes of the FDS/Smokeview documentation are also automatically tagged with this information on the title page of each document providing documentation and traceability of a given version of the documents to a specific version of FDS/Smokeview and a specific repository revision. Release notes indicate whether the changes affect a particular model feature. Maintenance releases are either just bug fixes or the addition of minor enhancements (such as a new output quantity), and should not affect code functionality.



\chapter{Software Requirements Identification, Design, Implementation, and Testing}

Changes are made to the FDS source code daily, and tracked via revision control software. However, these daily changes do not constitute a change to the version number. After the developers determine that enough changes have been made to the source, they release a new minor upgrade, 6.2.12 to 6.2.13, for example. This happens every few weeks. A change from 6.2 to 6.3 might happen only a few times a year, when significant improvements have been made to the model physics or numerics.

The software development process for FDS/Smokeview is intentionally simple. The process can be defined in five basic steps:

\begin{description}
\item [1. Identify and document the need for a change:] The need for minor changes to FDS typically come from user inquiries. These may be reports of bugs in the software or requests for additional features or outputs from the model.

    Significant changes to FDS/Smokeview, guided by available research road maps (see ) are made based on the following criteria, in no particular order:
\begin{description}
\item[Better Physics:] The goal of any model is to be {\em predictive}, but it also must be reliable. FDS is a blend of empirical and deterministic sub-models, chosen based on their robustness, consistency, and reliability. Any new sub-model must demonstrate that it is of comparable or superior accuracy to its empirical counterpart.
\item[Modest CPU Increase:] If a proposed algorithm doubles the calculation time but yields only a marginal improvement in accuracy, it is likely to be rejected. Also, the various routines in FDS are expected to consume CPU time in proportion to their overall importance (Baum's Rule\footnote{After Howard Baum, NIST Fellow and one of the original developers of the mathematical foundation of FDS.}). For example, the radiation transport algorithm consumes about 25~\% of the CPU time, consistent with the fact that about one-fourth to one-third of the fire's energy is emitted as thermal radiation.
\item[Simpler Algorithm:] If a new algorithm does what the old one did using fewer lines of code, it is almost always accepted, so long as it does not decrease functionality or accuracy.
\item[Increased or Comparable Accuracy:] The validation experiments that are part of this guide serve as the metric for new routines. It is not enough for a new algorithm to perform well in a few cases. It must show clear improvement across the entire suite of experiments. If the accuracy is only comparable to the previous version, then some other criteria must be satisfied.
\item[Acceptance by the Fire Protection Community:] Especially in regard to fire-specific devices, like sprinklers and smoke detectors, the algorithms in FDS often are based on their acceptance among the practicing engineers.
\end{description}
\item [2. Analyze, evaluate, and implement the change:] Algorithms for complex changes are typically documented by revisions to the FDS or Smokeview Technical Reference Guide and specific test cases to verify the correct operation of the model changes are included in the FDS or Smokeview Verification and Validation Guides. Software implementation then follows the equations included in the relevant Technical Reference Guide. Additional details of the process, tracking, and testing of changes is included in the sections below.
\item [3. Review and approval of the change:] Before accepting any change into the repository, verification and validation tests are run to ensure desired results. All changes to the software and documentation are automatically tracked in the FDS and Smokeview repositories.
\item [4. Verification and Validation:] A suite of simple verification calculations are run each night to ensure that the daily bug fixes have not altered any of the important algorithms. A suite of validation calculations are run each night. Any changes to the results are automatically flagged and emailed to developers for review. Additional details of the process is included in the sections below.
\item [5. Release of new versions:] NIST policy requires review and approval of technical reports (all the FDS/Smokeview documentation) and software (the FDS/Smokeview software released on the FDS website). Additional details of the review process is included in the sections below.
\end{description}

\section{Problem Reporting and Resolution}

NIST has developed an automated reporting and resolution tracking website for use with the FDS model to facilitate tracking and cataloging of inquires, problems, and model enhancements / revisions. This is included as part of the FDS website at \newline
\href{https://github.com/firemodels/fds/issues}{\textct{https://github.com/firemodels/fds/issues}} \\
\href{https://github.com/firemodels/smv/issues}{\textct{https://github.com/firemodels/smv/issues}}.

\section{Software Change Implementation}

Each developer works on various routines, and makes changes as warranted. Minor bugs are fixed without any communication (the developers are in different locations), but more significant changes are discussed via email or telephone calls. A suite of simple verification calculations (included in this document) are run each night to ensure that the daily bug fixes have not altered any of the important algorithms. A suite of validation calculations (also included here) are run routinely and always with each significant upgrade. Any change to verification or validation results are flagged and reported for further investigation to ensure changes in model outputs are expected with given changes to the model code.


\subsection{Creating a Change Request}

Change requests are submitted using the FDS Issue Tracker.  The Issue Tracker is an online service provided by GitHub. A change request is initiated by opening a new issue.  The issue report contains the baseline identification (version number, compile date, and revision number), operating system, and a description of the defect or enhancement request.  Input files or other supporting documentation can be attached. If the issue is opened by a user, it will be given a status of {\em New} until it is reviewed by a developer. This typically takes a few minutes to a few hours, depending on the time of day the issue is reported. If the issue is opened by a developer, the developer can immediately assign a status and an owner.


\subsection{Committing Changes}

Once a developer has addressed a change request, the modified files are committed to the FDS repository.  A description of the changes will be added to the change log.  This description first identifies the primary component being changed (for example: FDS Source or FDS Documentation).  This component identification will be followed by a brief summary of the changes made.



\section{Software and Peer Reviews}


As a general rule, all team members watch for unintentional commits of copyrighted material or algorithms, and also proprietary data or case studies collected from an end user. FDS itself does not include copyrighted or proprietary data, but occasionally this kind of information is submitted by an end user who has submitted a bug report.

\subsection{Reviews Prior to Software Release}

Prior to final implementation of a new feature or significant change, a review of the proposed modification and documentation is conducted by other members of the development team.  This review includes review and concurrence of the software requirements and design specification as well as more detailed review of code changes as the complexity of the modification requires. Review and acceptance of the software requirements and design specification by interested project sponsors or users may be included as appropriate. Name and date of approval and/or review is noted electronically in the NIST review process.

FDS is reviewed both internally and externally. All documents issued by the National Institute of Standards and Technology are formally reviewed internally by members of the staff. The theoretical basis of FDS is laid out in the present document, and is subject to internal review by staff members who are not active participants in the development of the model, but who are members of the Fire Research Division and are experts in the fields of fire and combustion. Externally, papers detailing various parts of FDS are regularly published in peer-reviewed journals and conference proceedings. In addition, FDS is used world-wide by fire protection engineering firms who review the technical details of the model related to their particular application. Some of these firms also publish in the open literature reports documenting internal efforts to validate the model for a particular use. Many of these studies are referenced in Volume 3 of the FDS Technical Reference Guide~\cite{FDS_Tech_Guide}.

\subsection{Survey of the Relevant Fire and Combustion Literature}

\label{Relevantdocs}

FDS has two separate manuals -- the FDS Technical Reference Guide~\cite{FDS_Tech_Guide} and the FDS User's Guide~\cite{FDS_Users_Guide}. The Technical Reference Guide is broken into four volumes: (1)~Mathematical Model, (2)~Verification, (3)~Validation, and (4)~Configuration Management. Smokeview has its own User's Guide~\cite{Smokeview_Users_Guide}. The FDS and Smokeview User Guides only describe the mechanics of using the computer programs. The Technical Reference Guides provides the theory, algorithm details, and verification and validation results.

There are numerous sources that describe various parts of the model. The basic set of equations solved in FDS was formulated by Rehm and Baum in the {\em Journal of Research of the National Bureau of Standards}~\cite{Rehm:1}.  The basic hydrodynamic algorithm evolved at NIST through the 1980s and 1990s, incorporating fairly well-known numerical schemes that are documented in books by Anderson, Tannehill and Pletcher~\cite{Anderson:1}, Peyret and Taylor~\cite{Peyret:1}, and Ferziger and Peri\'{c}~\cite{Ferziger:1}. This last book provides a good description of the large-eddy simulation technique and provides references to many current publications on the subject.  Numerical techniques appropriate for combustion systems are described by Oran and Boris~\cite{Oran:1}.  The eddy dissipation concept (EDC) combustion model is described in Poinsot and Veynante~\cite{Poinsot:TNC}. Basic heat transfer theory is provided by Holman~\cite{Holman:1} and Incropera~\cite{Incropera:1}. Thermal radiation theory is described in Siegel and Howell~\cite{Siegel:1}.

Much of the current knowledge of fire science and engineering is found in the {\em SFPE Handbook of Fire Protection Engineering}~\cite{SFPE} and the {\em Fire Protection Handbook}~\cite{NFPA}. Popular textbooks in fire protection engineering include those by Drysdale~\cite{Drysdale:1} and Quintiere~\cite{Quintiere:2}. On-going research in fire and combustion is documented in several periodicals and conference proceedings. The International Association of Fire Safety Science (IAFSS) organizes a conference every three years, the proceedings of which are frequently referenced by fire researchers. Interscience Communications, a London-based publisher of several fire-related journals, hosts a conference known as Interflam roughly every three years in the United Kingdom. The Combustion Institute hosts an international symposium on combustion every two years, and in addition to the proceedings of this symposium, the organization publishes its own journal, {\em Combustion and Flame}. The papers appearing in the IAFSS conference proceedings, the Combustion Symposium proceedings, and {\em Combustion and Flame} are all peer-reviewed, while those appearing in the Interflam proceedings are selected based on the submission of a short abstract. Both the Society for Fire Protection Engineers (SFPE) and the National Fire Protection Association (NFPA) publish peer-reviewed technical journals entitled the {\em Journal of Fire Protection Engineering} and {\em Fire Technology}, respectively. Other often-cited, peer-reviewed technical journals include the {\em Fire Safety Journal}, {\em Fire and Materials}, {\em Combustion Science and Technology}, {\em Combustion Theory and Modeling} and the {\em Journal of Heat Transfer}.

Research at NIST is documented in various ways beyond contributions made by staff to external journals and conferences. NIST publishes several forms of internal reports, technical notes, special publications, and its own journal called the {\em Journal of Research of NIST}. An internal report, referred to as a NISTIR (NIST Inter-agency Report), is a convenient means to disseminate information, especially when the quantity of data exceeds what could normally be accepted by a journal. Often parts of a NISTIR are published externally, with the NISTIR itself serving as the complete record of the work performed. Previous versions of the FDS Technical Reference Guide and User's Guide were published as NISTIRs. The current FDS and Smokeview manuals are being published as NIST Special Publications, distinguished from NISTIRs by the fact that they are permanently archived. Work performed by an outside person or organization working under a NIST grant or contract is published in the form of a NIST Grant/Contract Report (GCR). All work performed by the staff of the Fire Research Division at NIST beyond 1993 is permanently stored in electronic form and made freely available via the Internet.




\subsection{Review of the Theoretical Basis of the Model}
\label{JustAA}

The technical approach and assumptions of the model have been presented in the peer-reviewed scientific literature and at technical conferences cited in the previous section. The major assumptions of the model, for example the large-eddy simulation technique and the combustion model, have undergone a roughly 40 year development and are now documented in popular introductory text books. More specific sub-models, like the sprinkler spray routine or the various pyrolysis models, have yet to be developed to this extent. As a consequence, all documents produced by NIST staff are required to go through an internal editorial review and approval process. This process is designed to ensure compliance with the technical requirements, policy, and editorial quality required by NIST. The technical review includes a critical evaluation of the technical content and methodology, statistical treatment of data, uncertainty analysis, use of appropriate reference data and units, and bibliographic references. The FDS and Smokeview manuals are first reviewed by a member of the Fire Research Division, then by the immediate supervisor of the author of the document, then by the chief of the Fire Research Division, and finally by a reader from outside the division. Both the immediate supervisor and the division
chief are technical experts in the field. Once the document has been reviewed, it is then brought before the Editorial Review Board (ERB), a body of representatives from all the NIST laboratories. At least one reader is designated by the board for each document that it accepts for review. This last reader is selected based on technical competence and impartiality. The reader is usually from outside the division producing the document and is responsible for checking that the document conforms with NIST policy on units, uncertainty and scope. The reader does not need to be a technical expert in fire or combustion.

The US Nuclear Regulatory Commission (US NRC) published a seven-volume report on its own verification and validation study of five different fire models used for nuclear power plant applications~\cite{NUREG_1824_Sup_1}. Two of the models are essentially a set of empirically-based correlations in the form of engineering ``spread sheets.'' Two of the models are classic two-zone fire models, one of which is the NIST-developed CFAST model. FDS is the sole CFD model in the study. More on the study and its results can be found in Volume~3 of the FDS Technical Reference Guide~\cite{FDS_Tech_Guide}.

Besides formal internal and peer review, FDS is subjected to continuous scrutiny because it is available free of charge to the general public and is used internationally by those involved in fire safety design and post-fire reconstruction. The quality of the FDS and Smokeview User Guides is checked implicitly by the fact that the majority of model users do not require a formal FDS-specific training course, but are able to read the supporting documents, perform a few sample simulations, and then systematically build up a level of expertise appropriate for their applications. The developers receive daily feedback from users on the clarity of the documentation and add clarifications when needed. Before new versions of the model are released, there is a several month ``beta test'' period in which users test the new version using the updated documentation. This process is similar, although less formal, to that which most computer software programs undergo. Also, the source code for FDS is released publicly, and has been used at various universities world-wide, both in the classroom as a teaching tool as well as for research. As a result, flaws in the theoretical development and the computer program itself have been identified and corrected. As FDS continues to evolve, the user base will continue to serve as a means to evaluate the model. We consider this process as important to the development of FDS as the formal internal and external peer-review processes.

\section{Model Testing}

Individual testing of model algorithms are made by the developers during any revision of the model. Often, this testing forms the basis for any new standard test cases included with future model releases. System release testing of the model prior to release includes the following:

\begin{itemize}
\item Examination of results of test cases specific to any model modifications made as appropriate.  Comparison with analytic solutions to simplified problems is desirable when available.

\item Examination of results of standard test cases included with the release version of the model. Any changes in the output from prior versions is explained (and documented in the software testing and validation results report) by modifications made to the model.

\item For major new releases of the model, a complete suite of test cases should be compared to those from previous versions of the model.  At a minimum this includes the set of validation exercises described in U.S. Nuclear Regulatory Commission publication NUREG 1824 \cite{NUREG_1824, NUREG_1824_Sup_1}, but also include numerous additional example cases or validation exercises as appropriate.

\item For all changes to the model, an automated testing process is used that provides automatic error checking and collecting performance and accuracy statistics between FDS revisions. Results of this process for release version of the model are included in the Verification and Validation Guides for FDS \cite{FDS_Validation_Guide, FDS_Verification_Guide} and Smokeview \cite{Smokeview_Verification_Guide}.
\end{itemize}


\section{Firebot: Automated Software Quality Testing}

The FDS and Smokeview source codes undergo an automated build, verification, validation, and regression testing process, which is similar to a continuous integration system that automates the build/testing cycle for a project. This automated process is referred to as Firebot. The Firebot build/test process helps the FDS-SMV development team by performing automatic error checking and collecting performance and accuracy statistics between FDS revisions.

The automated built/test verification process runs on a regular schedule (nightly) to ensure that FDS and Smokeview are free of compiler errors, verification errors, or any other errors that would result in a failure during the build/test cycle. The automated build/test validation process runs continuously through the validation suite set-by-set. For future reference, the results of a build/test process are archived and tagged with the appropriate revision number that was used.

Upon completion of the automated build/test process, the results of the build/test process are dispatched to the development team. In the event of a failure, the build/test process continues when possible, and the appropriate error logs are dispatched to the development team. The automated build/test process consists of eight build stages, which are described in the following sections.

\subsection*{Stage 1: Checkout of FDS-SMV codebase}

In the first stage, the newest version of the FDS-SMV source code is retrieved from the FDS-SMV repository at GitHub. At this stage, the Firebot script ensures that all temporary files are removed and that a clean copy of the repository is used for the later stages, as if an end user were downloading the repository for the first time.

\subsection*{Stage 2: Compilation of FDS debug versions}

FDS is compiled with debugging flags enabled. This stage allows for errors in the source code to be identified and traced early in the build/test process. Any compiler warnings are logged at this time, and the build/test process continues after no major compilation errors are found.

\subsection*{Stage 3: Run verification suite (or validation set) in debug mode}

All of the cases in the verification suite (or validation set) are invoked using the FDS debug versions that were compiled in Stage~2. The verification (or validation) cases are run for at least one time step to ensure that the cases start without errors. After all of the cases run for at least one time step, the cases are stopped using an FDS stop file. Then the FDS output logs are checked for Fortran errors, numerical instabilities, missing verification case files, or other FDS errors.

\subsection*{Stage 4: Compilation of FDS release versions}

The release version of FDS is compiled. Any compiler warnings are logged at this time, and the build/test process continues after no major compilation errors are found.

\subsection*{Stage 5: Run verification suite (or validation set) in release mode}

All of the cases in the verification suite (or validation set) are invoked using the FDS release versions that were compiled in Stage~4. After all of the cases run to completion, the FDS output logs are checked for Fortran errors, numerical instabilities, missing verification case files, or other FDS errors. If validation mode is active and the simulations are free of errors, then the updated results are committed back to the repository.

\subsection*{Stage 6: Compilation of Smokeview and scripted generation of figures}

The Smokeview debug and release versions are compiled along with the Smokeview utilities (smokezip, smokediff, and background). Any compiler warnings are logged at this time, and the build/test process continues after no major compilation errors are found.

Additionally, scripts in the FDS and SMV verification suites are used to generate figures in Smokeview using automated Smokeview scripting commands. These figures are used in the FDS User~\cite{FDS_Users_Guide} and Verification~\cite{FDS_Verification_Guide} Guides as well as the SMV User and Verification Guides.

\subsection*{Stage 7: Generation of verification and validation plots and regression statistics}

Scripts in the FDS verification and validation suites are used to generate plots, regression information, and timing statistics using the results from Stage~5. These plots, figures, and statistical information are used in the FDS User~\cite{FDS_Users_Guide} and Verification~\cite{FDS_Verification_Guide} Guides as well as the SMV User and Verification Guides.

\subsection*{Stage 8: Compilation of FDS-SMV Guides}

The FDS User~\cite{FDS_Users_Guide}, Technical Reference~\cite{FDS_Math_Guide}, Verification~\cite{FDS_Verification_Guide}, and Validation~\cite{FDS_Validation_Guide} Guides, FDS Configuration Management Plan, and SMV User, Technical, and Verification Guides are compiled to ensure that all figures and equations can be compiled without any \LaTeX\ errors.


\section{Issuing New Releases}

The decision to change versions of the software is made by consensus of the development team, usually after it is determined that enough changes have been made to warrant a new release. There is no formal process to determine when a new release is to be issued. However, once the decision is made, the new version is given a number, it is tested, and then posted to the official download site.


\subsection{Testing a New Release}

Each proposed release undergoes testing. There are two sets of input files that are used to test new releases -- a verification suite and a validation suite. The verification suite consists of a range of calculations, from ones that just test a feature to ones that compare FDS results with analytical solutions of the governing equations. The validation suite consists of simulations of a wide range of experiments. The level of testing depends upon the type of baseline release: maintenance, minor, or major.

A maintenance release occurs every few weeks. Each maintenance release is tested with the verification suite during the nightly Firebot testing. Maintenance releases are intended for routine bug fixes, documentation clarification, and usability issues. There should be no changes in code functionality with a maintenance release, although the bug fixes might change results slightly.

A minor release occurs every three to six months. Each minor release is tested with both the verification and validation suites. The validation cases take considerably longer to run, but the procedure is similar -- the results are plotted and compared to the experimental measurements using an automated plotting package that essentially redraws all of the plots and graphs in the FDS Validation Guide~\cite{FDS_Validation_Guide}. The old and new guides are compared to assess whether or not the new version of the model has become noticeably less accurate than the old. The FDS~Validation Guide contains a table in the Summary chapter that lists the bias and relative standard deviation values for all of the predicted quantities. These are the uncertainty metrics that ought to be cited by anyone using this particular minor release. In essence, for a minor release, the assessment procedure outlined in ASTM~E~1355 is repeated. The verification and validation studies are redone by the automated continuous integration scripts.

A major release of FDS occurs every few years. As the name implies, major changes are made to the algorithm, requiring months of beta testing, and several runs through the verification and validation suites. It is expected that a major release will introduce, and potentially remove, major algorithms and functionality.


\subsection{Announcing a New Version}

Following successful completion of the required baseline testing, a new version is released.  Prior to release, the version identification information within the FDS source code is updated.  FDS documentation is updated to include the new version number.  The new version is compiled and new installation packages are uploaded to the FDS download site.  Prior versions are archived and deprecated.



\chapter{Software Risk Management}

The primary risk management tool for software developed and released by NIST is the official NIST review process for software, documents, and other products of NIST outlined above. Official approval is required prior to release of the model for general use. Additional processes to minimize risk are identified below.

\section{Media Control}

Release versions of FDS and Smokeview are available exclusively on the FDS specific website included in the FDS/Smokeview GitHub repository structure.

As part of its model development, NIST maintains a web-based system for version control and history of both the FDS and Smokeview source code and of pre-release and release executables for the software.

These computer systems are available only to specified personnel, including the FDS project leader and project team members.

\section{Supplier Control}

FDS does not include any commercial software libraries.

NIST currently uses Microsoft Visual Studio and Intel Fortran and C++ for development\footnote{Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose.}.  Prior to any change to a different development system, a full test suite of test cases are compared to verify consistent operation of the model and model results.

\section{Records Collection, Maintenance, and Retention}

In addition to the identified configuration items included in the FDS/Smokeview repositories, all software, documentation, and SQA documents are retained by the FDS project leader, typically in electronic form. Software and documentation is also maintained and archived on the NIST FDS website as part of the available older versions of the model.

NIST management approval is required prior to destruction of old or out-of-date records. Records are typically maintained for a minimum of 25 years.

\section{Training}

No specific training is identified for use of this plan.  Details of training requirements for use of the model included in the FDS user's guide is applicable to developers of the model as well.









\backmatter
\nopart

\bibliography{../Bibliography/FDS_refs,../Bibliography/FDS_general,../Bibliography/FDS_mathcomp}

\printindex

\end{document}
